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ABSTRACT 

This unobtrusive study conducted at the Brigham Young 
University Library was designed to determine whether reference 
questions are appropriately referred by student and paraprofessional 
assistants. Issues addressed included: (1) the percentage of 
questions assistants cannot answer themselves that are referred to 
professionals; (2) the percentage of questions more appropriately 
answered on another floor or department that are referred to that 
floor or department; (3) the percentage of questions requiring the 
patron to go to an outside collection that are referred to that 
collection; (4) the percentage of requests for documents not 
possessed by the library that are referred to inter library lean; and 
(5) the percentage of appropriate referrals by paraprofessional 
assistants compared with referrals by student assistants. The study 
population included 38 assistants at five reference desks. Eleven 
proxies asked the assistants a total of 125 reference questions over 
a 3-week period. These questions were designed to study the accuracy, 
negotiation, and referral aspects of reference service. Results 
indicated that the assistants are aware of their limits and the 
limits of the collections, as well as other collections in the 
library and on campus. Referral performance was impressive except for 
the lack of referral to interlibrary loan. Suggestions for further 
research are included. Referral questions, proxy instructions, the 
proxy report, and the data list are appended. (13 references) 
(MES) 
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APPROPRIATENESS OF REFERRALS 
IN HAROLD B. LEE LIBRARY: 
AN UNOBTRUSIVE STUDY 

CHAPTER 1 
THE PROBLEM 

In 1986, the Graduate Council and Faculty Library Council of 
Brigham Young University recommended to the Harold B. Lee Library 
Administration that the reference services be reorganized. The 
recommendation called for Student Reference Assistants (SRAs) and 
fulltime paraprofessional Department Assistants (DAs) to staff 
the patron service desks. Professional librarian Subject 
Specialists (SSs) were removed from the desks to attend to other 
duties, although they provided backup on a scheduled basis to the 
SRAs and the DAs as needed. 

In winter Semester 1987, a study was conducted to see how 
well the SRAs and DAs were performing at the reference desks 
(Hall, et al.). The original study consisted of five parts. The 
present study dealt only with the portion that concerned the 
appropriateness of referral given by the SRAs and DAs (hereafter 
referred to jointly as Assistants, except where distinction was 
deemed necessary). 

In the 1987 study, unobtrusive questions were designed and 
asked in order to determine the SRAs ' performance in the areas of 
accuracy, question negotiation, and referral. However, none of 
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the questions was specifically designed to measure referral 
performance. The researchers decided that any question an SRA 
could not answer or answered incompletely be came a referral 
question, and the answer was scored according to whether or not 
the SRA referred it to someone who could provide a complete 
answer. Of the seventy-five questions asked, it was determined 
that seventeen should have been referred. Eleven of the seven- 
teen (65%) were referred appropriately (Hall, et al. 1987). 

Statement of the Problem 

Are questions requiring referral appropriately referred? 

Since the 3 987 study (Hall, et al.), a policy and procedure 
manual ( Reference Services Manual ) and training programs for the 
Assistants were developed. In conjunction with the present 
study, the accuracy and negotiation portions of the previous 
study were replicated by other researchers to see what effect the 
manual and training have had. The manual and training also 
address referral, but the 1987 study was not systematic in its 
evaluation of referral practices. This study, therefore, 
attempted to provide a baseline measure of referral performance 
by the Assistants. This study was not a replication of the 
previous study, so a direct comparison of referral rates was not 
possible. However, this study was designed to be replicable, so 
that later studies can evaluate differences in the level of 
service over time or after other training or procedural changes 
are made. 
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Importance of the Study 
The measurement of referral performance is important because 
it indicates how well patrons are directed to the areas or people 
that can provide answers to their questions, and it can indicate 
what steps need to be taken to improve this part of reference 
service. A high percentage of appropriate referrals means that 
patrons are saved a lot of wasted time, effort, and frustration 
caused by seeking information in the wrong place. 

Literature Review 
There have been a number of studies of referr; 1 by librari- 
ans and other professionals to other non-library professionals or 
services, such as doctors, specialists, therapists, or social 
service agencies (Childers, 1984; Hawley, 1987). By contrast, 
studies of in-house library referrals or referrals to other 
libraries are rare. A study by Halldorsson and Murfin (1977) 
revealed that nonprofessionals referred or consulted on only 28% 
of those questions they were unable to answer. Murfin and Bunge 
( 1988 ) found that paraprof essionals on the reference desk only 
consulted other staff on 7.9% of reference questions. Their 
respondents reported much less satisfaction with the service they 
received from paraprqf essionals, primarily because of the 
latter *s inability to adequately negotiate queries and provide 
more complete answers. The paraprofessionals 1 weakness in 
negotiation resulted in their not discovering the patrons 1 real 
need and referring them to appropriate sources for help. Mere 
frequent consultation with or referral to professionals might 
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have helped the paraprof essionals learn better negotiating and 
answering skills, since the patrons rated the service by profes- 
sionals significantly higher. Van House and Childers (1984) 
examined the accuracy rate of five reference centers to which 
local libraries in the five systems referred questions they could 
not answer, but did not look at in-house referral. 

Hawley (1987) specifically addressed referrals within the 
library as well as to other libraries and pointed out the lack of 
published research in this area. He conducted an investigative 
study to determine factors that influenced whether or not 
librarians made referrals. He identified fourteen factors 
grouped in four categories, including Personal Factors, Library 
Factors, Outside Resources, and Factors Concerning the Users. 
Many of these factors are likely to influence the decision of the 
Assistants to make referrals, but some of the factors apply more 
to professional librarians in relatively permanent employment. 
Those whose performance was measured in this study have primarily 
their BYU library employment traininq and knowledge of library 
resources to encourage referrals. 

R esearch Questions 

The main question this study attempLed to answer is: What 
percentage of questions requiring referral are appropriately 
referred? 

Subsidiary questions are: 
1. What percentage of questions Assistants cannot 

answer themselves do they refer to professionals? 
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2. What percentage of questions more appropriately 
answered on another floor or department are referred to 
those floors or departments? 

3. What percentage of questions requiring the patron to go 
to an outside collection are referred to those collec- 
tions? 

4. What percentage of requests for documents not possessed 
by the Lee Library or BYU Library System are referred 
to Interlibrary Loan? 

5. How did the percentage of appropriate referrals by OAs 
compare with the percentage of appropriate referrals 
made by SRAs? 

Assumptions 

This study assumed that referral to the appropriate depart- 
ment, floor, or professional resulted in the patron's being able 
to get an accurate answer to his question. Since the profes- 
sionals are the highest level resource available, they are 
assumed to be sources of correct information. In order to find 
the answer, the patron must first find the collection or the 
individual that can provide the answer. 

It was also assumed that the Assistants working at the 
reference desks during this period did not differ in any 
significant or systematic way from those employed at any other 
non-holiday period, so the results should be generali2able to the 
service as a whole. 
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The study also assumed that the Assistants had read the 
Reference Services Manual and participated in the training 
program as required for each f loo* 

It was assummed that the Assistants were unaware of the time 
period when the data was collected. 

It was assummed that proxies understood and followed 
direct ions , both in asking questions and in completing the 
questionnaire. 

Delimitations 

This study was primarily concerned with referrals to on- 
cfimpu:? library facilities, although referrals to off-campus 
libraries occurred and were accepted. In practice, librarians 
refer patrons to places they feel could provide an answer, not 
necessarily to the closest or most efficient source (Hawley, 
1987), so a less efficient referral was not penalized if it was 
an effective one. 

The researchers did not address referrals to non-library 
professionals (doctors, therapists, etc.). 

The study also did not address the effect of maturation or 
experience on SRA performance, since the high turnover rate 
ensured almost 100% new employees since the previous study (this 
will likely be true between the current study and any successive 
studies , too) . DAs , on the other hand , may be employed for 
longer periods, so experience and maturation may be factors with 
them. 
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This study did not test the accuracy of final answers to 
questions, but only whether a referral to an appropriate resource 
was made. 

Definitions 

Accuracy: "Correct and complete answer (s) that satisfy(ies) 
a patron's information need." 

Department Assistant: "Department Assistants are full time 
paraprofessional staff hired primarily to supervise and assist in 
providing reference service. The amount and responsibility given 
to this person may vary by department, depending on the person's 
knowledge and experience in the department" ( Reference Services 
Manual , p. 3.18). 

Interlibrary Loan (ILL): A department maintained within the 
library which, upon request, provides patrons with documents 
borrowed from another library. 

Negotiation: Methods used by reference personnel to clarify 
the patron's ultimate information need. 

Other Floor/Department: A floor of the library other than 
the one the patron is on when asking his question, or a separate 
department or collection housed within the library but not within 
the control of the reference librarian or staff at the reference 
desk (i.e. Archives and Manuscripts or Special Collections, which 
have their own staffs to provide assistance). 



12 



8 

Outside Collection: A library collection rsually located on 
campus but not in the Lee Library building. Two questions fron 
the fourth floor call fov of f -campus referrals because the 
religion collection has no other on-campus resource. A partial 
listing of the on-campus collections can be found in the Site 
Description section of Chapter 2. 

Professional Librarian/Subject Specialist: Full-time 
librarians with MLS degrees and a specialty in one or more 
subject areas who have offices near the reference desks on their 
specialty floors and are available for consultation and special 
services on a scheduled or on-call basis. 

Proxies: Students in the LIS 696 class during Winter 
Semester 1989 who asked the questions and provided data about the 
interactions and the answers received. 

Referral: The ALA Glossary (1983) does not give a definition 
of referral. The Webster 's Third New International Dictionary 
(1981) defines referral as "the process of directing or redirect- 
ing to an appropriate specialist or agency for definitive 
treatment" (p. 1908) . Childer 's (1979) definition applies to 
Information and Referral Services, and specifies that referral is 
"facilitating the link between a person with a need and the 
service, activity, information, or advice outside the library 
which can meet the need" (p. 2036). This definition rules out 
most of the behaviors that this study sought to examine. Hawley 
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developed and refined his definition of referral throughout his 
book, finally deciding that referral is: 

An act of library employees . . . responding 
to individuals 1 needs by directing these 
individuals to another person, or to a place 
under the control of another person, for the 
fulfillment of these needs (Hawley, p. 144). 

This study used Hawley's definition because it includes all 
the activities the researchers wanted to study and encompasses 
the referral criteria given in the Lee Library's Reference 
Services Manual : "If [the Assistant] cannot meet tue users' 
needs, refer the questions or the users to the paraprof essional 
department assistant, the reference specialist, or the appropri- 
ate subject specialist (p. 3,18)." 

Hawley leaves Interlibrary Loan (ILL) out of the scope of 
his definition because it is carried cut by the librarian who 
takes the question. At the Lee Library, a patron may be sent to 
ILL, which is a separate department, under another person's 
control. Thus, referral to ILL also fits Hawley 's definition 
which was used in this study. 

Student Reference Assistant: BYU students who are trained 
to answer questions and assist patrons on a specific floor of the 
library. They may or may not have a library background, although 
many of them are students in the MLIS program. 

Unobtrusive study: A study in which the subjects are 
unaware of being tested. 
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CHAPTER 2 
METHODOLOGY 



The purpose of the study was to measure the Student 
Reference Assistants' and Department Assistants 1 (collectively 
referred to as Assistants) performance in appropriately referring 
questions received at the reference desk. An unobtrusive test 
using proxies and questions designed to elicit referrals was 
developed to measure referral performance. 

Specifically, this study measured the service at JiYU's Lee 
Library, and data were gathered at all five reference desks in 
this library. During a three-week period, March 3-22, proxies 
asked selected questions of the Assistants at pre-determined 
reference desks and recorded the responses they received on the 
Proxy Question Report (see Appendix C). Eight questions designed 
to elicit referrals were asked on each floor, each question being 
asked only once. The responses were coded, tallied, and analyzed 
using SPSS/PC+ Studentware . 

Site Description 
The Harold B. Lee Library is a central library located in 
the middle of the BYU campus. Because two floors are below 
ground-level, the main floor is actually the third floor. Five 
floors house the collection, and each one has a reference desk 

10 



15 



11 

located near the middle of the floor, very visible and accessible 
in high traffic areas near the stairway and elevator entrance to 
the floor. These desks are staffed by Assistants, 

The floors are divided according to subjects as follows: 
First floor ; Ethics, Government Documents, Social 
Sciences, Education, Sociology, and Psychol- 
ogy (also, Map collection and Asian collec- 
tion) : 

Second floor : General Science, Physical Science, 
Home Economics (also, Learning Resource 
Center) ; 

Third floor : General Reference (also, Reserve 

Reading Room and Interlibrary Loan); 
Fourth floor : History, Religion, Philosophy, 

Anthropology (also, Genealogy collection and 

the Special Collections room); 
Fifth floor : Humanities, Music, Art (also, 

Archives and Manuscripts and Juvenile 

collection) • 

A computerized catalog (BYLINE) has terminals on all floors 
and is accessible by personal computers both on- and off-campus. 
BYLINE has records of books acquired since 1978 plus older items 
circulated since then, and a card catalog on the third floor 
contains holdings up to 1978. Computer terminals are also 
available on most floors to perform searches on RLIN and to 
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perform subject-specific searches on CD-ROM systems. 
Professionals perform online searches. 

In addition to the Lee Library, there is also a law library, 
business library, small science library in the Bean Museum, music 
library, and several specialized learning resource centers on 
campus • 

The Library's collection includes approximately 2 million 
volumes and the annual circulation is about 800,000. 

Population 

The population included thirty-eight Assistants who worked 
at the five reference desks in the Lee Library during the weeks 
of March 3 to 22, 1989. There is one DA working on each floor, 
for a total of five. Based on the Winter 1989 work schedule, 
there were seven SRAs on the first floor, six on the second 
floor, seven on the third, seven on the fourth, and six on the 
fifth floor. Four SRAs, two from the first floor and two from 
the third floor, belonged to the group of MLIS students 
performing this survey, and they were excluded from the sample 
tested. These four students did not ask questions on the floors 
on which they work . Using a photographic directory, work 
schedule, and the Assistants 1 name tags, the researchers 
attempted to include as many members of the entire population in 
the study as possible. The Winter semester extends from early 
January to late April, and the sample period was close to 
midterms. This time period should have provided a representative 
sample of reference services. 



17 



13 



Data Collection Procedures 

Proxy training ; The proxies included the eleven students (5 
males, 6 females) in LIS 696 class in the Winter semester 1989. 
All of these students have had the research methodology course 
offered in the MLIS program ( one was taking it concurrently) . 
All eleven students participated in the development or critiquing 
of questions, coding sheets, and questionnaires. Several class 
sessions were devoted to discussion of how to handle and score 
various types of responses to questions. Each student acted as a 
proxy in a local public library's reference services study in 
February, 1989. Successes and problems from that experience were 
used to refine research methodology, questions, and the Proxy 
Question Report and instructions. An instruction sheet was 
included with the scoring sheet to remind the proxies of how to 
ack questions and score various responses (see Appendix B). 

Data collection ; The schedule for data collection was 
developed by examining the work schedules for each of the five 
floors. The researchers isolated thirty-three SRAs from the 
schedules. DAs were not listed, and professionals and the four 
SRAs who were in LIS 696 class were not included. The 
researchers selected times when each SRA worked alone, if 
possible. For those who did not work alone, a time was selected, 
when they worked with one of the SRAs from the LIS 696 class. 
This was to attempt to ensure that each SRA was asked at least 
one question. No attempt was made to control for SRAs trading 
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shifts. However, there were five-to-seven SRAs working on each 
floor, so the remainder of the eight questions ppr floor were 
scheduled at times when two or more were working at the desk. 
The DAs are full-time workers, available during regular office 
hours (8 a.m. to 5 p.m. ) , and could be asked questions at any 
time. Approximately half of the questions were asked before neon 
and the other half between noon and 5 p.m. No questions were 
asked after 5 p.m. or on weekends because many offices and 
departments were closed at those times, making referral much more 
difficult and, therefore, less likely. 

The proxies selected questions to ask on the basis of the 
proxy's availability at the time the questions were scheduled to 
be asked. A total of 125 questions (from three studies running 
concurrently) were distributed in this manner, so any particular 
student may have received one to seven referral questions in 
their total of approximately thirteen questions each. They were 
allowed to exchange questions if they did not feel they had the 
background to answer any negotiation questions the Assistants 
might have asked. Proxies were instructed to ask the questions 
as given, except that they could substitute "My husband {wife, 
friend, etc.)" in place of "I" if the Assistant knew the proxy 
and would know the premise of the question was false. The 
referral questions were designed in such a way that no further 
negotiation or background information should have been needed. 

Proxies asked their questions over the three-week period, at 
the specified location, at the specified time and day of the week 
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in all but five cases. Immediately after the interaction, away 
from the reference area, the proxy completed the Proxy Question 
Report attached to the question, A photographic directory was 
used to confirm the status of the staff involved in the 
transaction. 

At the end of the test period, all the questions and score 
sheets were collected. Answers to questions designed to study 
one aspect of reference services (accuracy, negotiation, and 
referral) that fit more than one of these categories, were 
analyzed in each of the appropriate studies. For example, forty 
questions were designed for referral, but since the negotiation 
study group analyzed all of the usable questions, both referral 
and negotiation groups evaluated the results. In the previous 
study, those questions that received a wrong or incomplete answer 
were treated as questions that should have been referred. The 
researchers in this study felt Assistants would not give an 
answer they knew to be wrong. Therefore, the Assistants could 
not be expected to refer a question they thought they knew the 
answer to. 

The researchers in this study evaluated the referrals given 
to see if they were made appropriately. The professional 
librarians in charge of each of the floors provided the most 
likely or appropriate sources, but referrals that differed from 
those sources were checked by the researchers. The alternate 
source or the librarian in charge of the libraries or collections 
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was contacted by phone or in person and askad whether or not the 
answer could be found in that collection. 

The Proxy Instructions and Proxy Question Report are 
included in Appendices B and C. 

Instrument 

The library literature failed to disclose a survey 
instrument appropriate to this study. Outside of the library 
field there were many studies of referral behaviors, but they 
included only referral to sources outside of the library, and 
usually to non-library professionals, especially counselors or 
medical specialists. For these reasons the researchers created a 
new instrument. In consultation with the subject specialists on 
each floor, eight questions from each floor were developed. 
These consultations attempted to ensure that the questions were 
realistic, clear, and varied in degree of difficulty so that some 
would require professional assistance. The forty questions were 
then typed and given back to the librarians for review during 
their monthly staff meeting so that they could make sure that the 
level of difficulty for each floor was approximately equal . 
After this review the questions were returned to and critiqued by 
the LIS 696 class members. The Lee Library Reference Evaluation 
Committee reviewed the questions and made some changes in 
consultation with the subject specialists, again in an attempt to 
make the questions very similar in type and degree of difficulty 
for each floor. The revised questions became the instrument. 
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The questions were open-ended and designed to cover four 
types of referral: 

1. to a professional librarian; 

2. to another floor or department; 

3. to an outside collection; 

4. to Interlibrary Loan. 

The questions are reproduced in Appendix A. 

The instructions given to the proxies are in Appendix B, and 
Appendix C comprises the Proxy Question Report . The Proxy 
Question Report was developed by the LIS 696 class to cover all 
three studies (accuracy/ negotiation and referral). Three 
referral questions were pretested to determine the adequacy of 
the Proxy Question Report , but the pretest was done too late for 
changes to be made in the form before data collection. However, 
proxy instructions were clarified on the basis of pretest 
experiences. 

Limitations 

This was meant to be an unobtrusive study; however, some of 
the Assistants were aware beforehand that it was to be done. 
Many of the SRAs are in the MLIS program and know the proxies 
personally. These factors worked against the study being 
unobtrusive. Another problem was that MLIS students could be 
expected to know the answers to many of the questions asked in 
the study. This became a major problem as proxies reported 
difficulty asking "stupid" questions and getting blank or 
surprised stares from SRAs who thought the proxies should know 
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the answers to those questions* (In at least one case, a proxy 
reported having changed the question to make it seem less 
"stupid, 11 but the other proxies stated they did ask the questions 
as given. Even though the intent of the question was changed the 
answer still resulted in a correct referral. Question #R3.4 
"Where is CUE ?" was changed to "I need to know what indexes 
index CUE .") It may have been obvious to the Assistants who 
knew the proxies that these were test questions. However, few 
Assistants had access to the dates of the testing period, and 
many of the proxies ask questions at the reference desks fairly 
regularly, so these factors may have made it harder for the 
Assistants to differentiate actual queries from cest questions. 

The small number of questions asked on each floor made it 
impossible to adequately compare the proportion of accurate 
referrals on the different floors. 
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CHAPTER 3 
ANALYSIS OF DATA 

Forty questions were designed to elicit referrals from the 
Student Reference Assistants ( SRAs) and Department Assistants 
(DAs ) f at the Lee Library reference desks. Eight questions were 
asked on each of the five floors. Each set of eight questions 
consist 3d of four types, with two questions of each type. The 
four types were referral to 1) a professional librarian, 2) 
another floor or department, 3) an outside collection, and 4) 
Interlibrary Loan. Two other studies, testing negotiation and 
accuracy, were done concurrently, using the same proxies as this 
study. Any questions from those two studies that were referred 
or were not answered and, therefore, should have been referred 
were added to this study. 

Four of the referral questions were unusable. Three that 
were designed to be referred to a professional were answered 
correctly by SRAs without requiring referral (#3.1, 3.2, and 
4.1), one question ( Interlibrary loan referral, 3.7) was 
mistakenly asked of a professional. Ten questions from the 
Accuracy and Negotiation studies were referred, and were adtfed to 
this study. 

The forty referral questions were scheduled to be asked on 
specified floors, days, and times, in order to test all floors 
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equally and to try to have each SRA or DA respond to at least one 
question. All questions were asked on the correct floor and 31 
of the 36 usable questions were asked in the specified hour, so 
most of the population was tested. The most frequent reason for 
questions being asked at the wrong time was that a professional 
or a student who was also a proxy in the study was manning the 
desk at that time because of schedule changes or other 
uncontrollable reasons. 

The SRAs' and DAs' appropriate referral rates were to be 
compared, but this was not possible since only 3 DAs provided 
answers to any of the 46 questions (including the 10 Negotiation 
and Accuracy questions) asked. That sample is far too small to 
be meaningful, so the SRAs and DAs were grouped together in the 
final results. 

Since the data was nominal, only simple frequencies, 
percentages, and ratios were calculated, using SPSS/PC* 
Studentware . Only descriptive, not inferential statistics are 
given. (The data file is included in Appendix D.) 

If a partial answer was given without referral to a source 
that could provide a full answer, then the question was scored as 
"answered incorrectly." If an answer was essentially, "I don't 
know," and no referral was given, the answer was scored as "no 
answer, no referral." 

The percentage of questions appropriately referred was 
obtained by comparing the number of questions that should have 
been referred to the number of questions that were appropriately 
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referred. Eighty percent of the referral questions were 
appropriately referred. with the additional ten questions from 
the Accuracy and Negotiation studies , 85% were appropriately 
referred. 

The percentage of appropriate referrals to a professional 
was obtained by comparing the number of questions that should 
have been referred to the number of questions that were ap- 
propriately referred to a professional. One hundred percent of 
all these questions were appropriately referred. 

The percentage of appropriate referrals to another floor or 
department was obtained by comparing the number of questions that 
should have been referred to the number of questions that were 
appropriately referred to another floor or department. Ninety 
percent of all these questions were appropriately referred. The 
question inappropriately referred in this category was R5.3. 

The percentage of appropriate referrals to an outside 
collection was obtained by comparing the number of questions that 
should have been referred to the number of questions that were 
appropriately referred to an outside collection. Eighty percent 
of all these questions were appropriately referred . The 
questions inappropriately referred in this category were R1.6 and 
R2.6. R5.5 was referred to a professional. 

The percentage of appropriate referrals to Interlibrary Loan 
was obtained by comparing the number of questions that should 
have been referred to the number of questions that were ap- 
propriately referred to Interlibrary Loan. Fifty-six percent of 
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all these questions were appropriately referred. The questions 
inappropriately referred in this category were R2.7, R3.8, R4.8, 
and R5.7 . R3 .7 was mistakenly asked of a professional and was 
excluded from the study. 

Seven Accuracy and three Negotiation questions were 
refer red . Fou r of them were appropr iat ely referred to 
professionals. Five questions were appropriately referred to 
another floor or department. One question was appropriately 
referred to an English professor . Overall, 100% of these 
questions were appropriately referred. 

Table 1, below, summarizes the above data. 

Again, this is a baseline study. Successive studies will be 
able to test for significant changes in the proportion of ap- 
propriate referrals. 



Table 1. — Percentages 


of Appropriate and Inappropriate Referrals 


Type of Referral 


Number of 

Usable 

Questions 


Percentage of 

Appropriate 

Referrals 


Percentage of 
Inappropriate 
Referrals 


Professional 




7 


100% 




Another floor 
or department 




10 


90% 


10% 


Outside collection 




10 


80% 


20% 


Interlibrary Loan 




9 


56% 


44% 


Referred Accuracy/ 
Negotiation Questions 


10 


100% 
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CHAPTER 4 
SUMMARY AND CONCLUSIONS 



The present study attempted to measure what percentage of 
questions requiring referral were appropriately referred. The 
subsidiary questions were: 

1. What percentage of questions Assistants could not 
answer themselves were referred to professionals? 

2. What percentage of questions more appropriately 
answered on another floor or department were referred 
to those floors or departments? 

3. What percentage of questions requiring the patron to go 
to an outside collection were referred to those collec- 
tions? 

4. What percentage of requests for uocuments not possessed 
by the Lee Library or BYU Library System were referred 
to Interlibrary Loan? 

5. How did the percentage of appropriate referrals by DAs 
compare with the percentage of appropriate referrals 
made by SRAs? 

This study was intended to be unobtrusive, using LIS 696 
students as proxies for collecting data. The data collection 
involved the use of 40 referral questions (Appendix A). The 
questions were scheduled to be asked at predetermined times and 
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places. At the end of the *iree-week data collection period, all 
questions and score sheets w e collected, distributed to each of 
the groups (Accuracy, Negotiation, and Referral), and the data 
was analyzed using SPSS/PC+ Studentware . 

The 1987 Hall et al. study produced an appropriate referral 
rate of 65%. The current study was designed to look at referrals 
more systematically, and the overall appropriate referral rate 
was 85%, with a range from 56% for "Referrals to Interlibrary 
Loan" to 100% for "Referrals to a Professional." The latter was 
somewhat inflated by definition; any question sent to a profes- 
sional was considered properly referred, since the professionals 
are the highest level resource available and are assumed to be 
sources of the correct information. Even with this weakness, the 
results were impressive. Five of the questions designed for a 
referral to a professional were written with a particular 
professional in mind. In four cases, the Assistant gave precise- 
ly that professional's name (the fifth was answered without 
requiring referral) , so these were instances of knowing the 
proper resource. Only one of the questions designed to be 
referred to a specific place was referred to a professional 
instead, indicating that the Assistants were not turning over a 
majority of the questions to the professionals. Four additional 
referrals to professionals came from the Accuracy and Negotiation 
questions. [These questions were numbers A1.3 , A2.8 , A4.6 , N5.1 . 
The following questions also were refer red : A3 . 5 , A3 . 6 , A4 . 5 , 
A4.9, N3.1, and N3.5 (Beck, Carter, and Skousen 1989; Follett, 
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Giles, and Wright 1989).] Three of the questions designed for 
professional referrals were correctly answered by the Assistants 
without referral and were not included in referral statistics. 
This indicates that the Assistants did even better than expected 
by the professionals who assisted in designing the referral 
questions • 

Questions designed to measure referral to another floor or 
department were referred to the correct area 90% of the time 
demonstrating that the Assistants are familiar with areas other 
than their own. 

The researchers expected that performance would be much 
poorer in referrals to outside collections, feeling the Assis- 
tants would be less familiar with resources outside of the Lee 
Library. However, the Assistants were able to refer the proxies 
to an appropriate outside collection 80% of the time. 

Questions needing referral to Inter library Loan were 
appropriately referred only 56% of the time. The researchers had 
expected these questions to yield the best results. Proxies 
reported being told, ,! We don't have (the requested document)," 
which is basically what the proxy had told the Assistant in the 
wording of the question. Apparently, the Assistants were not 
familiar with the services that the Interlibrary Loan provides 
and should be reminded of how to deal with requests for materials 
not in Harold B. Lee Library's holdings. 

The questions that were added to the study from the Accuracy 
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and Negotiation groups were appropriately referred 100% of the 
time. 

The overall results of this study indicate that the Assis- 
tants are very much aware of their own limits and the limits of 
their collections. They are also aware of the other collections 
in the Lee Library and on BYU campus. Their performance in 
redirecting proxies to proper sources that could answer their 
questions was very impressive. The one weak area in the referral 
performance was in the lack of referral to Interlibrary Loan for 
documents not owned by the Lee Library. 

Recommendations 

Due to the excellent results obtained in the study, the only 
recommendation as far as improving training, is to emphasize that 
patrons who request documents the library does not possess should 
be referred to Interlibrary Loan. 

Several recommendations can be made, though, on ways to 
improvo the study and possibly improve its accuracy and reliabi- 
lity. 

Suggestions for Improving the Present Study 
1. Folxowing the 1987 Hall et al. study, the Reference 
Services Manual was developed and a training program implemented 
at the Harold B. Lee Library. Each floor has adapted the 
procedures from the manual as applicable to their staff training 
needs. This study assumed that all Assistants had read the 
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manual and completed the training. Future studies might verify 
this rather than assume it. 

2 . The size of the instrument was fairly small, limiting 
the interpretation of the resulting data. Future studies could 
include more questions for each category. Questions to be 
referred to collections outside the Lee Library should include 
all resource collections to ensure that Assistants are knowledge- 
able with all other on-campus collections. In designing addi- 
tional or replacement questions, it is important to ensure that 
the level of difficulty for each floor be kept approximately 
equal. This was the most difficult part of developing the 
instrument in this study, but it would be especially important if 
a future study attempted to compare performance of the different 
floors with each other. Such comparative studies would enable 
the librarians to ensure a consistent level cf training on each 
floor . 

3. The main fault in this study was the fact that the 
referral questions were asked by LIS 696 students who were peers 
of many SRAs working at the reference desks. Some of the 
questions were inappropriate for # LIS students to be asking, while 
others were difficult because the SRAs knew that the premise of 
the question was false (Example: 2.2: "I am a school librarian 
..."). It i3, therefore, recommended that future studies do not 
use library science students as proxies. Each student could find 
a proxy (friend, roommate, spouse, etc.) who could come to the 
training session and to one session after the data is gathered to 
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report any problems and whether they followed instructions on 
each question. 

4. There were several problems with the Proxy Questior 
Report . The same report form was used for three different 
studies, which resulted in some confusion in recording informa- 
tion about the interaction and the answers received. In combin- 
ing forms developed by each group, questions were moved around, 
context was lost, and the form ended up being less useful to all 
three research groups than their original forms. For example, 
the checklist labeled "Question type" was misinterpreted to mean 
what group the question came from rather than how the question 
was dealt with. Also, question #7, "Indicate the way in which 
the reference assistant dealt with your question" did not give 
the option "Referred." Question #8 "Write the answer given to 
the question here:" would have been clearer as "Write the answer 
given to the question, or the person or place referred to, here." 
A separate form should be used for each study in the future. It 
might also be helpful to have a role-playing session during the 
class period to simulate some of the possible interactions , 
followed by a discussion of how to record the results on the 
Proxy Question Report . 

Suggestions for Further Research 
1. The definition of 'referral* needs to be standardized in 
the Harold B. Lee Library. It was discovered in the course of 
developing the instrument that the different professionals within 
the Lee Library had a variety of definitions that affected the 
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questions they developed* Some considered any appeal to another 
person or place as a referral, while others felt that passing 
question to a DA or a professional was just following the chain 
of command, and the act was not a referral unless the patron was 
sent to another floor, department, or outside collection. This 
complicated the gathering of questions designed for referral to 
professionals. 

2. Further studies of referral performance could develop 
questions testing various aspects of referral as this one did. 
The types used here may not be appropriate, but other areas, such 
as referral to public resources, government agencies, or crisis 
services could be included. 

3. This study gathered data only during weekdays and 
business hours. Future studies may choose to measure referral 
performance on Saturdays and evenings as well. 
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APPENDIX A 
THE INSTRUMENT: REFERRAL QUESTIONS 



First Floor 

Referring to a professional librarian 

Rl.l I'm having difficulty finding information on the 
political theorists and theories of the early 20th 
century. Is there someone who can help me find some 
information on this? [Referral to Larry Benson] 

R1.2 I've looked on ERIC and Social Science Index but need 
to do a more extensive search in preparation for a 
Master's thesis in geography. Isn't there some kind of 
in-depth search service you have? [CARS search, Larry 
Benson] 

Referring to another floor 

R1.3 I'm considering a trip to Europe this summer. I found 
your maps, but do you have any travel guides that would 
tell me more about the area? [Travel guides on 4th 
floor] 

R1.4 I have to do a report on this American psychologist, 
Richard Patrick Vaughan, and the teacher said the 
information was here, but I can't find him in the 
International Encyclopedia of Psychology, Psychoanalys- 
is, and Neurology , or Corsini 's Encyclopedia of 
Psychology , or International Encyclopedia of Social 
Sciences . Can you tell me where to look? [3rd floor, 
American Men and Women of Science: Social and Be- 
havioral Sciences ] 

Referring to an outside collection 

R1.5 I need to know about the mental health laws in Califor- 
nia (like what are the requirements to have a person 
committed against his will). Where do I look for them? 
[Law library] 
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R1.6 I've already looked at the Occupational Outlook 
Handbook , but do you have anything that explains 
nursing more carefully? [ LRC , Kimball Tower] 

Referring to Interlibrary Loan 

R1.7 I can't find this book in the card catalog or Byline, 
can you tell me how I might be able to borrow it? — 
Absent Mandate: the Politics of Discontent in Canada by 
Clarke . ~. . et al. Toronto, Canada: Gage, 1984. 
[ILL] 

R1.8 I found some references to articles in the Calcutta 
Journal of Political Science from Political Science 
Abstracts. I can't find thit journal listed on the 
microfiche or BYLINE and I'd really like those ar- 
ticles. Is there any way I can get hold of it? [ILL] 



Second Floor 
Referring to a professional librarian 

R2.1 I need to do an online search on comparative effective- 
ness of different schizophrenia medications for my 
Master's thesis. How do I do this? [Richard Jensen] 

R2.2 I am a school librarian and need to get some recommen- 
dations of good basic science reference books to buy 
for my collection. [Referral to science reference 
librarian] 

Referring to another floor 

R2.3 I need some statistics on alcoholism in the U.S. [1st 
floor] 

R2.4 I need some information on Linus Pauling, the scientist 
who thinks vitamin C will cure the common cold. 
[Biography reference — American Men and Women of 
Science] 
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Referring to an outside collection 

R2.5 I want to mount the head of the next deer I shoot , but 
the books I have found in your book stacks are all very 
old. Do you have something more recent? [Bean Museum 
has more recent books on taxidermy] 

R2.6 Where can I view a videotape on robotics engineering? 
[Crabtree LRC] 

Referring to Interlibrary Loan 

R2.7 I need this article in Bulletin of the American Academy 
of the History of Dentistry 36: 31-36 (1988 ) but I 
couldn't tell from the microfiche if you had it or not, 

R2.8 How can I get hold of a book you don't have? I'd like 
to get a copy of the second edition of Numerical 
Methods for Scientists and Engineers by R. W. Hamming, 
(New York: McGraw-Hill, 1973) 



Third Floor 
Referring to a professional librarian 

R3.1 I need the name and address of a newspaper currently 
published in Cleeve, England, so I can place a genealo- 
gical ad, [Referral to professional librarian] 

R3.2 Can you give me an address to write to for information 
on this summer's National Old-Time Fiddlers' Contest in 
Weiser, Idaho? [Referral to professional librarian] 

Referring to another floor 

R3.3 I need a copy of The History of the Church . [4th floor 
or Archives] 

R3.4 Where is CUE? [1st floor. Current Index to Journals in 
Education ] • 

Referring to an outside collection 

R3.5 I need current information on the IBM Corporation. 
[Business library in Tanner building]. 

R3.6 I found this book on BYLINE but I cannot find it in the 
library. The book is Writing without Teachers by Peter 
Elbow, (Call number 808.042) Where can I find it? 
[McKay LRC] 
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Referring to Interlibrary Loan 



R3.7 We do not have this journal, Immunology Today . The 
librarian on level 2 said I could get it from another 
library. How do I do that? 

R3.8 BYLINE says The Twilight Zone: The Origin? 1 Stories 
should be in the Sampler but that it is lost. Is there 
some way I could get a copy of it? 



Fourth Floor 
Referring to a professional librarian 

R4.1 This book says "Do Not Circulate" but is there any way 
I could take it out just for overnight? (The title of 
the book is Fireside and Devotional Speeches, 1981-1983 
(BX 8647 .B76J) [See Susan Fales to check out or Gary 
Gillum from Ancient Studies] 

R4.2 What is the Biblical Hebrew word for 'love 1 and how is 
it related etymologically to its New Testament Greek 
equivalent? [Referral to Gary Gillum] 

Referring to another floor 

R4.3 Do you have a tape of a Devotional talk given by 
President Benson? [2nd floor LRC] 

R4.4 I have found your books on Spanish history, but where 
can I find books on political science in Spanish? [1st 
floor] 

Referring to an outside collection 

R4.5 Do you have a picture of Christ being baptized that I 
can use in my Relief Society Lesson? [Ward library] 

R4.6 I got this call number, 974.721 H2t, from the GLC and I 
can't find it here. The title is History of Long 
Island; from its Discovery and Settlement to the 
Present Tim e. [Family History Library, SLC] 

Referring to Interlibrary Loan 

R4.7 I'm looking for a book called Germany Must Perish by 
Theodore N. Kaufman, written in 1941, but I can't find 
it in the card catalog or on BYLINE. How can I get it? 

R4.8 Where would I find the book The Gospel According to 
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Pontius Pilate by James R. Mills? It's not in the card 
catalog or on Byline. 



Fifth Floor 
Referring to professional librarian 

R5.1 I am beginning preliminary work on my master's thesis 
in English and need to find out which topics the BYU 
library has sufficient material on. [Professionals are 
charged with conducting thesis interview with graduate 
students. ] 

R5.2 I am looking for information on Mormon film makers. 
[Blaine considers this a professional question] 

Referring to another floor 

R5.3 I am looking for a review of a biography about C. S. 
Lewis. The title is Clive Staples Lewis: A Dramati c 
Lif e , by William Griffiru (Book review indexes are at 
General Reference.] 

R5.4 I need some information on the philosophy of dance. 
[Dance is classified in GV on Level 1.] 

Referring to an outside collection 

R5.5 I need about 20 copies of some music that I can use 
with my BYU Ward choir. [The Music Performance Library 
in the Harris Fine Arts Center has performance scores.] 

R5.6 I have to prepare lesson plana for teaching English 
literature for my student teaching and I need some 
recent hiqh school English textbooks. [McKay LRC has a 
collection of recent textbooks.] 

Referring to Interlibrary Loan 

R5.7 I have been looking for an article from Vol. 7 of the 
journal Critique (PN 3503 .C7), but I can never find it 
on the shelf. How can I get a copy of the article? 

R5.8 I'm trying to get a book by Barry Hannah, but the 
library doesn 1 1 seem to have any. How can I get a 
copy? (Possible titles Geronimo Rex and Nightwatchmen . 
The library has ordered Ray but has not received It 
yet.) 
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APPENDIX B 



PROXY INSTRUCTIONS 



The library is presently evaluating the quality of the 
reference service provided at the reference desks. An important 
part of this evaluation is to determine the correctness and 
completeness of the answers given, and the ability to negotiate 
and refer skillfully. While we are not attempting to evaluate 
the work of specific staff members, it is important that none of 
the reference desk staff know that they are being tested so they 
will not change their usual behavior. 

During the time period of March 3-22, you will be given 
approximately 12 questions to ask at each of the assigned five 
reference desks at specified times in the HBLL . Immediately 
following each reference transaction, please fill out the Proxy 
Question Report to report the results. Do not wait to fill out 
the forms or you might forget important details. It is important 
that these forms be filled out with care and deliberation. 

The questions you have been assigned are coded so that each 
question is asked at the appropriate reference desk: 

A 1.1-1.12 

R 1.1-1.8 Social Science Reference Desk , Level 1 

N 1.1-1.5 



A 2.1-2.12 

R 2.1-2.8 Science Reference Desk, Level 2 

N 2.1-2.5 



A 3.1-3.12 

R 3.1-3.8 General Reference Desk, Level 3 

N 3.1-3.5 



A 4.1-4.12 

R 4.1-4.8 History Religion Reference Desk, Level 4 

N 4.1-4.5 



A 5.1-5.12 

R 5.1-5.8 Humanities Reference Desk, Level 5 

N 5.1-5.5 



Please ask at the assigned desk, even if you think it should 
be asked elsewhere. 
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We have attached each question onto a Proxy Question Report 
form. Please do not detach these, 

1, Before asking each question, review the Proxy Question 
Report form. Be certain you understand the types of 
questions asked on the form so you will be able to 
answer them correctly, 

2, Be sure you fully understand the question you ask so if 
the SRA/DA asks you for further clarification you can 
provide an intelligent response. Do not suogest types 
of sources or places where the answer may be obtained: 
let the SRA/DA make the suggestions. The answer to the 
question is provided to you so you may respond intel- 
ligently, and the answer should not be used to prompt 
the SRA/DA, 

3 , We have endeavored to provide a rationale for why you 
are asking for the information in each question, but if 
there is not a specific purpose specified, indicate 
that you need the information for a class assignment or 
to write a paper, 

4, Try to act as normal and natural as possible. Do not 
allow the SRA/DA to see the paper with the question on 
it or the report form. You might find it useful to 
write the question or take a few notes from it on a 
piece of paper so you can show it to the SRA/DA, should 
you be asked. This could help you with call numbers, 
spelling, etc. Fill out your report form away from the 
reference desk area. 

5, Do not argue or be persistent in trying to obtain an 
answer , Take the answer given you even if it is 
incor rect , If an answer cannct be obtained , do not 
prompt or help the SRA/DA with clue* or with sugges- 
tions , 

6, If the SRA/LA tells you to iind an answer on your own 
rather than giving a correct answer , negotiation, or 
referral, consider that as ai incorrect answer. Do not 
go back and ask for more help, 

7, If you are asked to come back later when someone more 
knowledgeable can help you, consider that as a referral 
to a professional and do not return. If the SRA/DA 
offers to take you to speak directly wit.i the 
professional librarian, make an excuse that you cannot 
take the time just now and say that you will come back 
later. Also, if the SRA/DA consults with the profes- 
sional without you, count that as a referral to a 
professional, 

8, Avoid visiting a reference desk if a professional is 
present. If the SRA/DA refers you to a professional 
librarian who happens to appear at the desk during the 
time of your query, or if you are taken to the profes- 
sional , continue to play your part until you can 
explain to the professional that you are part of the 
reference evaluation study team, and record it as a 
referral to professional. Do not divulge this informa- 
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tion in the presence of the SRA/DA. if you are 
referred from one Assistant to another/ record that as 
a consultation, and ask the question of the other 
Assistant and fill out the Proxy Question Report on the 
basis of the answer given. 

10. Do not score the professional librarian (SSs) in any 
way. 

11. If you are referred to a specific librarian (by name), 
at another floor, department, or outside source, score 
it as a referral to a professional. If you are told to 
see "the librarian at McKay LRC," though, score this as 
a referral to an outside source. 

12. If the SRA/DA on the third floor gives you an Inter- 
library Loan request form to fill out, count that as a 
referral to ILL and take the form with you explaining 
that you'll return it later. The other floors do not 
have this form and will refer you either to ILL or the 
third floor desk for forms. Either one will count as a 
referral to ILL. 

13. If the SRA/DA is able to answer your referral question 
without a referral, fill out the Proxy Question Report 
based on the answer, reference techniques, and sources 
he used. It will not count as a referral, but will be 
included in the other studies that are using the same 
report sheet. 

14. When you have asked all of your questions and completed 
your reports, return the questions and report forms 
intact to your supervisor. Thank you very much for 
your time, effort, and willingness to help us improve 
our service. 



Instructions for question 12-18 on Proxy Question Report ; 

1. Questions 12 and 13 are subjective questions. Respond 
according to your own perceptions. 

2. Question 14: Open questions are broad and allow for 
dialog. Typically, they begin with What, Why, How, 
etc. 

3. Question 15: 'Verify' means verbal indication by the 
SRA/DA that the question has been understood, i.e., 
clarifying, paraphrasing, or by other methods. 

4. Question 16: SRA/DA verbally encourages follow-up to 
answer given by asking questions such as: "Does this 
answer your question?" or by making statements such as: 
"If you need more information, let me know." or "Come 
back if you need more help." 

5. Question 17 refers to the question asked in Step 2 of 
the escalator questions. 

6. Question 18 refers to the question asked in Step 3 of 
the escalator questions. 
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APPENDIX C 



Question Type: Accuracy 

(Check all Referral 

that apply) Negotiation 

PROXY QUESTION REPORT 

Question 



1. Your name 



2. Question asked on (Circle one): Level 12 3 4 5 

3. Question was asked when? Day Time 

4. To whom did you ask the question? SRA (1) DA (2) 

SS (3) Unknown (4) 

5. Reference Assistant or DA answered the question: 

a) (1) Without assistance. 

b) (2) After consultation with: 

SRA DA SS 

other unknown 



6. Check the following statement that BEST describes what 
you saw as you. approached the reference desk for help: 

a) (1) The SRA or DA was free to help me. 

b) (2) The SRA or DA was preoccupied with a 

library project, and I had to get their 
attention. 

c) (3) The SRA or DA was helping another 

patron at the desk, and I had to wait 

minutes for help. 

d ) m (4) The SRA or DA was away from the 

reference desk, and I had to wait minutes 

for help. 

e) (5) Other (specify): 



7. Indicate the way in which the reference assistant dealt with 
your question. 

a ) (1) Answered it without consulting reference 

sources, i.e. , books, computers, files, etc. 
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b) (2) Consulted a single reference source and found 

the answer. 

c ) (3) Consulted several sources and found the answer, 

d) (4) Showed you the reference source(s) that would 

answer your question and explained how to use it. 

e ) (5) Referred you to specific source(s) to find the 

answer on your own. 

f ) (6) Indicated that they could not find an answer. 

g) (7) Other (specify) 



8. Write the answer given to the question here: 



PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPppppppppppp 
(The following question to be completed by accuracy group) 
The answer given was: 

a) (1) Correct. 

b) (2) Partially correct. 

c) (3) Incorrect. 

d) (4) No answer or "I don't know". 

PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 

9. Did the SRA/DA cite the source used? (1) Yes (2) No 



10. Write the name of the source(s) used here: 



11. Approximately how long did you consult with the 
SRA/DA? Minutes. 

THE REFERENCE PERSON: 

12. Displayed a friendly and YES (1) NO (2) 
helpful manner during the 

reference interview. 



13. Used eye contact during 
reference interview. 



14. Asked open questions. 



15. Verified question 

during reference interview. 



16. Offered opportunity for follow- 
up. 



Question Negotiation Proxy Only: (17 and 18 only) 
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17. Was level 2 information 
discussed? 



18. Was level 3 information 
discussed? 



19. Referred question: to whom? (Circle or check one) 

a) SS (Subject specialist) yes (1) no (2) 

b) Other floor? first 

second 

third 

fourth 

fifth 

]?PPPPPPPPPPPPPPPPPPPPP?PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPppp 
(Tho following question to be answered by referral group) 
Correct floor? yes(l) no(2) 

PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 

c) Outside collection: 

Law library 

Bean Museum 

Tanner Business Library 

Kimball LRC 

McKay LRC 

Crabtree LRC 

HFAC Music Performance Library 

Other (specify) 

PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 
I (The following to be answered by referral group)! 
I Appropriate referral? Yes (1) No (2) [ 
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 

d) Interlibrary loan Yes (1) No (2) 

e) Other (specify) (9) 

PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 

(The following to be answered by referral group) 
Appropriate source? Yes (1) No (2) 
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 
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APPENDIX D 



DATA LIST 



Data list / 
Questgrp 1 (a) 
Floorask 2 
Questno 3 
Questype 5 
Staftitl 6 
Treatmnt 7 
ReftoSS 8 
ToPloor 9 
ToOut 10 
ToILL 11 
Toother 12 
Approp 13. 

Variable labels 

Questgrp "Question group" 

Floorask "Floor asked on" 

Questno "Question number" 

Questype "Referral question type" 

Staftitl "Staff title" 

Treatmnt "Question treatment" 

ReftoSS "Referral to professional" 

ToFloor "Referral to other floor/department" 

ToOut "Referral to outside collection" 

ToILL "Referral to Interlibrary Loan" 

Toother "Referral to other" 

Approp "Appropriate referral". 

Value labels 

Questgrp 'A 1 •Accuracy' 'R' 'Referral' ' N' 1 Negotiation ' / 
Questype 1 'Professional' 2 'Floor or dept • 3 'Outside 
collection' 

4 'Interlibrary loan' 0 'Does not apply' / 
Staftitl 1 ' SRA' 2 'DA' 3 'SS' 4 'Unknown' / 

Treatmnt 1 'Answered incorrectly' 2 'No answer, no referral ' 3 
'Referred' / 

ReftoSS ToFloor ToOut ToILL ToOther 1 'Correct or appropriate' 2 
•incorrect' 

0 'Does not apply' / 
Approp 1 'Appropriate' 2 ' Inapproprie te' . 

42 
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Begin data. 
Rll 113100001 
R12 113100001 
R13 213010001 
R14 213010001 
R15 313001001 
R16 311000002 
R17 413000101 
R18 413000101 
R21 113100001 
R22 113100001 
R23 213010001 
R24 213010001 
R25 313001001 
R26 313020002 
R27 412000002 
R28 413000101 
R33 223010001 
R34 213010001 
R35 313001001 
R36 313001001 
R38 421000002 
R42 113100001 
R43 213010001 
R44 213010001 
R45 313001001 
R46 313001001 
R47 413000101 
R48 411020002 
R51 113100001 
R52 113100001 
R53 212000002 
R54 213010001 
R55 313100001 
R56 313001001 
R57 411000002 
R58 413000101 
A13 013100001 
A28 013100001 
A35 023010001 
A36 013010001 
A45 013010001 
A46 013100001 
A49 013000011 
N31 013010001 
N51 013100001 
N35 013010001 
End data. 

Save outfile = 'A: REFERRAL. SYS ' . 
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